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Development of USES Specific Aptl'tu<le Test Battery S-282R.75 

«> 

• For ' / ' ^ 

■ V ' ■ . . . . ' 

Nurse Aid (medical ser. ) 355.878 

RESEARCH SUMMARY 
" , • . ■ .. 

This, report describes the research which resulted in the develop- 
ment *of the following Specific Aptitude Test Battery for us.e in 
selecting inexperienced or untrained individuals^ for training as 
Nurse Aids.:* • , . ' ^ . 

Cutting Scores ^ 

'80 • 
70 • 

80 > . , . 

Validation sample: 136 femaTe Nurse Aids enrolled In MDTA 
courses fn the South (see Appendix 2). A 'total of 68 were 
minority group members. (66'B1ackS/ 1 American lndlan//<and 1 
Spanish Surnamed) and 68 were nonmlnoritpy group members. 

Cro<$s-va1 1 d|it Ion *samp1e iJL: 155 iFemale Nurse Aids erhployed In 
hospitals In the Nprth and West (*see Appendix 2). This study was 
conducted prior to the requFrement of providing mlnorlty^group 
Information. Therefore^ minorl ty ^roup status of sample members 
I S' unknown. ' ^ • . ' * ' 

Crpss-val.l dajt ion sample '#2: 199 Nurse Aids (1.85 females and \k 
males) employed In 'hospital's In the West (see Appendl x 2). ' This 
study was conducted pclor to the requlremerit^'of providing 
mlnorjty^group Information. Therefore^ nflnorlty group statues of 
sample members Is unknown. • * 



Criterion^ i . ' o * 

VaUdatlon sample: Instructors' ratings. Criterion data v^ere 
cpflected durlnf^ 1971. 

Cross-validation sample #1:- Ra^nk ord«r ratings converted to line 
scores were obtained for ^0 samp\e members In 1955. and broad cate 
'gory ratings were obtained for 115 sample members In 195U. 

CrbsiS-'Val I dat ion sample #2: Supervl sory .rati ng^. Crltei^lon data 
were collected during 1962 and 1963. 



Apt I tudes 

G- General Learning Ability 
P - ForjTi Perception 
Q'- Clerical Perception 



• 2 - 



Des|gn ! , • . * V . 

VaUdatton sample; . V 

Longitudinal and concurrent* Test data were collected for 83 sub- 
jects at the beginning of training arid for 53 suj^jects at the end 
of trainings Criterion data were collected dt the end of the nine • 
weeks' training course, 

Gross-val Iddtlon samples^ * . ^ 

Concurrent, Test and criterion data were collected at about the. ^ 
, same time* ' ♦ ' \ 

V^alMltv t • . ^ ' \ ^ 

, Validation Sample: " ^ . 

Phi coefficient for total sample * .k2 (P/2 < ..0005') 
Phi coefficient for Black subsampl e • \37 (P/2 < ^005) 

, Phi coefficient fonnonmlnorl*ty subsample « .36 CP/l ^.005) 
1 ^ . ^ • ; 

/Cross-validation Sample #1: . • " » • • 

P,hl coefficient for 'total sample •',21 (P/2 < .01) • 

Cross-val Idailon sample #2: ' ^ 

Phi coefficient for total sample » .22 (P/2 < .p05) 

Comparison of Minority and Nonmtnqrltv Gro^osg / 
No differentjal validity was founa for tnis battery. The differ- 

' ence between \he phi coefficients for Black and nonmlnorlty groups 
for the val Idat lon^^ample Is, not statistically significant (CR ■ .11). 

^ The battery Is fa I r to^ Bl acks since the percent of -Blacks who met 
the cutting scores approximated the percent who were In the high 
cr I terlon ^groiip;' 58i of the Blacks met the cutt I ng scores and 61% % 
were In tKe high criterion group. 

JOB ANALYSIS 

Auob anc^lysls was performed by observation of the Nurse Aids* per- 
formance on tKe job and In donsultatlon with the Nurse Aids* super- 
visors. On the basis of tKe Job anal ysl s^ the job description 
shown In Appendix k was prepared^ Which was used to. (1) select . ^ 

• an experimental sample of Nurse Aids who were, performing the job 
duties; (2) choose an appropriate criterion or mea^sure of job per- ' 
for'mance;' (3) determine which aptitudes are crt tlcail^ Imppftant; or / / , 
Irrelevan^t to job performance (see Tables 1 and ii^/V and (i). provide / f 
Itiformatlon on, the applicability of the test battery resulting ' 
from th i'S -riBsearch." * • ^ . \ ; w\ /- 
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TABLE 1,. 
Qualitative Analysis 



G - General learning Abl 1 f t^ 



P - Farvn Perception 

0 

Q - n^ijl'cat Perception 



K' • Motoi^ Coordination 



M 



Manual Dexterity 



Rationale 

Required ^o learn and apply 
nursing techniques end hospital 
procedures; to. -comprehend 
written and verbal Instruct Ions» 

Required to perc;eiye dJf^erences 
between and defects in Instru- 
ments* 

Required to read and recor.<i 
accurately temperature^ pulse 
rate*/ respiration rate and blood 
pressure and ^to maintain stock 
.recorcjs. 

Required to take blood pres- 
sure and pulse rate readings* 

Required tO/ handle food tr^S/^ 
make betls and perform' clearing 
tasks* 



EXPERIMENTAL TEST BATflRf^^ 

12 tests of the GATtf^ B-1002B/ were administered to the vali- 
dation sample^and to x;ross-val Idat Ion sample #2 and all 12 tests 
of the GAYB/ B1002A/ were administered to cross-validation.. 

sample #1. ' * ^ 

«• * 

CRITERIA 

V'aHdation Sample: 

The Immediate instructor rated each trainee. The ratings were 
obtained bymeans of personal visits of State/test development , 
arialysts.who explained^ the rat? I ng ^procedure to the Instructors* 
Two ratings were obtained from each Instructor with an Interval * , 
of at least two Weeks between the ratings* Since sample members* 
test scores are conf I dent lal ^ instructors had no knowledge of the * 
test scores of . the trainees* * » 

A cl^s(?r i pt ive rating scale was used« The scale (isee Appendix^ 3) 
consists of seven Items* Six of these Items cover different as- 
pects of /jobS)er^formance. The seventh\Item ts. a g+obal item on the 



Nurse*A(d's "al 1 -around'V,ab.1 1 1 ty. Each I tern has five alternative 
responses corresponding to different degrees of job proficiency* 
For the purpose of scoring the i tems^ we! ghts of T ta 5 were ^ 
assigned to the responses* The tbtaT score on the rating scale Is 
the sum of the, weights for the seven items.. The possible range for 
each rating Is 7 - 35. - 

A review of the job description indicated that the subjects covered 
by the 'rating scale were directly related to^ Important aspects of 
job performance. . 

A - FaciUty: Nurse Aids perform a A^ariety of duties and musf 
learn each quickly ^nd, thoroughly. ♦ . 

B - Quantity of work: Nurse Aids must work quickly and effl- 

. ciently in providing patient care so that all patients needing 
care will receive attention* . . 

C - Qu'allty of work: Nurse Aid^' work must be of good quality to 
prov^ide optimum health care and to avpi d potent lal dangers of 
Improper treatment* ' ' ' 

D - Spejpd of learn?ng: Nurse Aids shcftild learn new duties with a 
\ minimuh of instruction and supervision* * 

E - AbilJty to use equl pmant : Equ i pment used^to t^k^ temperature ^ 
and blood pressure must be used properly. 

F vlob versatility^: Nurse Aids must be able to perform a variety 
of routine duties usuch as care of bed pa^ierrts^ assembling and 
sterilizing packs of supplies/ dresslnjgs alrr4 J nstrlimenf s^ arrang 
Ing suppl ies"*^ In stock rooms/ schedul I ng"" cl I n7^ appol ntments • 
for out-pat?entS/ and asslstlfng In post-morterrK o5ire . J 

^ t \ »' 

G - "All-around" job ability: Nurse A^ ds'" value *tb the health 
"care facility Involves a combination of as0ects bf job 
performance listed abovf. - t * . 

A reliability coefficient of .93 wai obtained between the initial 
ratings and the re-ratlngS/ Indicating a significant relationship. 
Therefore^, the final criterion Score consists of the fombln^ 
scores of the two ratings* The possible Vang^ for the final cri- 
terion Is Ik ^ 70. The actua>* rang€5/ 1 s 18-70^^ The mean Is k9.9 
with a standard dev lation pf ,11 .3. Thg' relatlojishlp between t*he 
criterion and age and educatJon^ fs shown Tn^ Table 2* * 




TABLE 2 . 

, Means/ Standard Deviations (SO) '^nd* Pearson 
Product-Moment Correlations with the Criterion (r)* for 
Age^ Education and Experience 

4 * « * 

, . ' Validation Sample 

' • * . . ' Mean' 50. ' r " . 

Age (years) 29.8 13.2 .12 

Education (years) 10.3 'l.? -.09 

About one-third of the workers are considered to be marginal 
workers. Therefore^ the criterion distribution was dichotomized 
so as to include about one-third of the sample in th£ low cri- 
terion, group *and the remainder In the high criterion group. The 
criterion cutting score was set at which pladfes 29% |.n the low • 
criterion group and 71% In the high crltef-lon grou'p. It was not 
possible tp place preci sely .one-thl rd of the workers in the low 
crltei^lon group because of the nature of the criterion distribu- 
tion.' ^ ' ^ ' 

Cross-validation Sample #1; . « ' 

The Immediate supervisor rated each worker. The ratings were 
obtained by means of personal visits of State test development 
analysts who explained the' rating procedure to the supervisors. 
Since, sample members' test scores are ,conf 1 dent la I ^ supervisors had 
no knowledge of- the test scores of the workers. 



The criterion for the Utah subsample consisted of supervisory 
ratings In the following four br^^d category job performance 
groupings: (1) outstanding^ (2) above average^ (3) average, and 
(h) below average. Two separate ratings were obtained for ea^rh 
worker with an interval of t>^o weeks- between the ratings. ^ the two 
ratings when combined yleld^^d seven levels of proficiency. The^e 
seven broad categories were converted to normalized standard scor& 
of 65, 59, 53, M, 33 and 29. ' , ' . 

The crlter^on' fq^r the Pennsylvania subsample consisted of super- 
visory rank-order ratings (see Appendix 3)\ The final criterion 
scores consisted of the rank-orlfer ratings converted to linear- 
scores. The relationship between the criteria and age, educf^t-lon 
and job experience is shown In Table 2a. * \ 



TABLE 2^a . 

Mesns^. Standard Deviations (SD) and Pearson 
Product-Moment Correlations wHh[ Supcrvl sory Ratings^ (r) 
for Age^ Education aipd Experience 

rross-val I dat Ion Sample #1 



^ " Age (y^ars) 31.8 11.8 -.15 .OH 

^ Education (years) 10.9 1*7 .02 .11 

Total Experience 18.8 20.3 .25' .07 

(months) • ' : * , 

♦Pennsylvania Sample. N«U0 
♦♦Utah Sample N-115 

About one-third of the workers are considered to»be marginal 
workers. Thi^refore^ the cri terlonyil strlbutlon wa? dichotomized 
so as to include about one-third of th^ sample In the low cri- 
terion group and the remainder In the ^hlgh criterion group. The 
cutting scores 09 tlus criteria were set at^ ^18 for the Utah sample 
and at ^il for the pAinsylvania sample which placed 30% of the Utah 
and, 52 1 of the Pennsylvania In low criterion groups aijd ,70* and 
68t If) high criterion groups^ respectively. Wheri combined^ 313; 
were In the low criterion group and 69t In the high criterion group 
It was not poss-lble to place precisely one-third of the workers In 
the low writer Ion group because of the nature of the criteria dis- 
tributions. ^ .1 

Cross^calldatlon Sample *1\ 

The immediate supervisor rated eac^ worker. The ratings were 
obtained by means of personal visits of State test development 
analysts who explained the racing 0rQcedure to the supervisors^.* 
Two ratings were obtained from each supervisor with an Interval 
of at least two weeks between .the irat lng<». Since sample members*' 
test scores are confidential^ supervisors had no knowledge of the 

test scores of the workers. ' ^ 

. ' * • 

A descriptive rating scale was used. The scale'(see Appendix 3) 
ppnslsts of nine Items. Eight of these Items cover different as- 
pects of job performance.^ The ninth Item Is a global Item on the 
Nurse Al d's 4'al 1-around'* ability. Each Item has five aFternatlve 
responses corresponding to different degrees of job proficiency. . 
For the purpose of scoring the ttems/ weights of 1 to 5 were 
assigned to the responses. The total score on the rating scale !s 
the sum of the weights for the nine Items. ^ A review of thfc job 
description revealed that the subjects covered by the rating scale 
were directly related toMmportant aspects of job performance.' 



A - Quantity of w.rk: Nurse Aids mus^.work quickly and efficiently 
In provldlhg patient care so that all patients needing care 
will receive attention. 

B* - Quality of work: " Nurse Aids* work must be of goo*d quality to 
provide optimum health care and to avoid potent lal dangers of 
\ Improper treatment. 

C - Accuracy of work: Nurse Aids must be able to detect accurately 
vital slgn^.such as changes In pulse rate^ /temperature r.eadlngs 
and blood pressure readings. * ^ 

D - Job knowledge: Nurse Aids must have sufficient knowledge of 
procedure^ to provide appropV I ate health care and to recognize 
when assistance of a professional nurse or phys4clan 1^ re^ ^ 
quired. , * 

m 

E - Facility; Nurse Aids perform a variety of duties and must 
learn each'^qulckly and thoroughly. t 

» * ' . * 

F - Job versatility: Nurse Afds must be able to perform a variety 
of routine duties such as care of bed patients^ assembling and 
sterilizing packs of suppl les^ ^dressl ngs and Instruments^ arrang 
Ing supplies In stock rooms^ schedul lng< cl Inic appointments 
for out-pat I en-ts^* and assisting In pOst-mortem care. 

G ^- Job resourcefulness: Nurse Aids must be able to react 
effectively In emergency s'ltuatlons. 

ri - Job Initiative: Nurse.^Alds ,shduld be able^ to suggest ways of 
providing services more effectively; 

I - "All-around" job ability: Nurse Aids' value tp the health 
care facility Involves a combination of aspects of job * 
performance listed above.. 

A reliability coefficient of .S6 Was obtained between the InUlal " 
ratings and the re-ratlhgs^ Indicating a significant. relatlonVhIp. 
Therefore^ the final criterion score consists of the cbmblned'X. 
scores of the two ratings. The possible range for* the final crl\ 
terlon Is 18 - 90. The actual range Is i»0-90. The mean Is 6^.6j 
with a standard >devlat Ion of 9.9*. The relationship between the - 
criterion and age^ education and job experience Is shown In Table 2b. 

r . 



4 

10 




4 
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TABLE ?.b 



/ 



Means, Standard Deviations, (SD) and Pearson 
Product-Moment Correlations with the Criterion (r) for 
Age/ Educatlbn and Experience 

\ Cross-validation Sample *2 , • 

- , Mean ill x , . 

, Age tytears) 39.6 11.2 .O'O 

Education (years.) 11.6 ;l.I5 .02^ " ,• • 
JE*perIence (months) -65.7 ?9.3 -.06* 

SAMPLE ' . . 

* s 

Validation Sample: ' ^ 

The valjdation s.ample consisted of 136 femal es enrol led in MDTA 
Nurse Aid courses in the Soyth (see 'Append! x» 2)". A total of* 68 were 
minority group members (66 Blacl<:S/ 1 American Indian^ and 1 Spanish 
Surnamed) and 68 were nonminority group members^ * The^means and 
standard devLatio^'ns for age^ education dnd^ exper'ience of sample 
members are shown in Table 2, Descriptive statistics for subgroups 
are shown in Appendix !• 

Cross-validation sample #1 : 15? females employed as Nurse Aids in 
hospitals in the North and West (see Append.ix 2)« This study wa^ 
conducted prior to the requirement of provldi ng minori ty group 
information. Therefore/ minority group status-of santple members 
•is unknown. All workers had at lea$t '1 month of job experience in 
Jobs with duties ^t^mifar to those shown in the Job description in 
Appendix The means and standard deviations^^for age^ education, 
anti experience of sample members are shown in Table 2a. 

Cross-val Idat jon sample #2- 199 Nurse Aids (185 females and m 
males employed In hospitals In the West. Th 1 s' study was con** 
ducted prior to the requlremejnt of provldi ng ml ngrl ty group In- 
formation. < Therefore^ minority group status of sample members Is* 
UQknown. The means and standard deviations for age^ education and 
experjence of sample members are shown In Table 2b. All workers 
had at 'least 3 montlis of Job experience In Jobs wjth dtitles similar 
to those shown in. the Job^descr I ptlon In Appendix 
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. ' ^ATISTICAL RESULTS 
' ' - TABLE 3 • 

V 

Statistical Results for VaUda^ion Sample 
\ ' • K-136 







Aptitude 






X 


G 




General LearnI ng Abl 1 1 ty 


86.it' 


.13.. 0 


.36** 


V 




Verbal Aptitude 


89.6 


10.5 


.35»* 


N 




Numerical Aptitude 


. 88.2 


16.1 


.2l»** 


S 




Spatial Aptitude 


•89.3 


15.6 


. .2l»** 


P 




Form Perception 


• 97.3 


18.7 


.21* 


Q 




Clerical Perception 


106.8 


1U.3 


. 28'** 


K 




Motor Coordination' 


105.0 


19.8 


-.08 


F 




Finger Dexterity 


98. .6 


20.3 


.21* 


M 




^Manual Dexterity 


98.6 


25.0 


-.05 



significant at the .05 level ^ * 

Significant at thef .01 level 

Table h summarizes the qualitative aihalysis and statistical results 
shown In Tabte.s' X and 3 and shows the aptitudes considered for In- 
clusion In the battery. * • 



/ 



12 



• ' ■ • . * ,- 10 - . 

. , TABLE k 

/iSuftifitary of Quail taA-ive and Quantitative Data for Validation Sample 

] ] . ^ * Aptitudes 



1 ' . . Type of Evidence 


G V > 


N 


S 


P 


Q 


K 


F 


M 


'/ / •'Critical!' on Basis > 
' of Job Anfi lysis 


4 
















' ^ ^' "Important" on Basis 
•Of Job AnaTysl»s 


X • ' 






X 


X 


X 




X 


, - "Irrelevant" on Basils 
of Job Anal ys is . 


JlelatlvQly High*' 
' Mean ; 










X 


X 


X 


X 


Relatively Low Standard 

' ^ Deviation 
•» ♦ 


X "X. 








X 








SI g^nf f tcajjt 6brrelation 
with (JrlteYlon 


X X 


' X 


X 


X 


X 




X 




Aptl.tudes Consider^ foY 
Inclusion In; the Battery 




N 


S 


p 


Q 


K 


F 


M 



The Information JnJTable h Indicates that the follpv/Ing aptitudes 
should bf cons4der'ed"Tt>r^4ndluslon In tKe battery: G/ V, S^ P, 
Q/ . and M. The objectTve^Js to develop a battery of 1, 3/ or 



k aptit;udes with.cutting scores set^-SLtfive poin-t Intervals at the 
point (a) where about the same percent wHJ^jrneet the- cutting scores 
as the percent placed In the h^gh cri ter Ion grd^ and (b) which 
will ma:)i;lmize the relationship between the batl^y 4nd the 
criterion. The cutting, scores are -set at approximately one 
standard deviation below the mean a0,titude scores of the sample^ 
with deviations above or below these points to achieve the objej:^:^ 
tives *^lndlcated above. 

The following battery resulted:, . * • 

Aptitudes Cutting Scores 

G - General Learning AbMlty 80 
P - Form Perception ^ 1^ 

Q - Clerical Perception . . . 80 ^ ' . • 
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VALIDITY OF THE BATTERY 
TABLE 5 

Validity of Battery for Total- Validation Sample 

^ * « 

Below " Mee*lri« 
Cutting Scores Cutting Scores Total 

High Criterion 20 77 97 

Gjvpup 

LflSif Criterion 25 Ik 39 / 

Group * ' ' 

Total 1*5 . - 91 136 

Phi coefficient « .l»2 
Significance level » P/2 .< .0005 

TABLE 5a 

Validity of Battery for Black Validation Subsample 

Below Meeting 
Cutting Scores Cutting Scorps Total' 

High Criterion 11 29 kO 

Group 

Low trl terl on 17 9 26 

Group 

Total 28 38 . 6,6 

Phi coefficient » .37 * * 

Significance level « P/2 < .005 



TABLE. 5b 

Validity of Battery for Nonminority Validation Subsampl 

Below Meeting 
CuttlnX Scores Cuttlng^cores Total 

High Cri^terion 8 i*8 56 

Group . . 

Low Criterion 7 5 * 12. 

Group / ' ' . • 

/ Total .15 53 68 

Phi coefficient « .36 (Yates* corrected) 
Significance level « P/2 < .005 
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• ' TABLE .6 

Vatldlty of Battery for Cross-vail dat| on Sample^'l 

Below Mefetl^g 
Cutting Scores Cutting SWes ISLtal 
-- V / . 

High Criterion , " 36 11/' 107 

Group ' ' - . ' / . 

Low Criterion 27 /21 " . ^8 

Gro.up ' /' . . ' ■ 

Total 63 <^ 92 . 155 . 

Phi coefficient « .21 * . " ' . • ' 

significance level » P/2 < .01 

V ■ - TABLE 7 ; 

Validity of Battery for Cross-validation Sample #2 

Below Meeting 
, ^Cutting Scores Cutting Scores l2iai 



-^Hlgh Criterion 36 V.. 96 - 132 
^- Group 

Low Criterion % 33 , . 3i» - 67 * 

Group ■ • , . 

Total 69 - . 130 199 " 

♦ 

Phi coefficient « .22 » .■ : . • . 

Signl-fic^nce level » P/2 < .005 * \' ' 

— ' . OCCUPATIONAL APTITUDE PATTERN 

This occupation was Incorporated into 0AP-2i» In Sectlon 11 of the 
1970 edition of t-hp' Manual for the USES Gene ral^ 'Apt.1 tude Test. 
Battery with a "Rouble asterisk" (*♦), becaui^e the cuttinig score for 
Aptitude P Is more than 10 points lower than the cutting score for , 
Aptitude p. in 0AP-2i» but a slgnificaVit phi- coef f I clen't wrfs obtained 
between the criteria and 0AP-2i» cutting scores of G-80, P-85 and 
Q-90. A phi coefficient of .28'(P/2 < .0005) was obtained for the 
combined validation sample and cross-validation samples. 
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APPENDIX 1 



i Descriptive Statistics for Black and Nonmf norl ty Subgroups 
. , ' of Va'1 1 datlon 'Sample - ■ i 



Black 
■( N«66) 



Nonmlnorl ty 
(M-6 8r ' 



Variable 

* 


Mean 


SSI 


RariK . 


Mean 




R^nKe 


Aptitude 6- 


. 8Z.3 


12.5 


61- 


112 


90.5 ^ 


12. H 


63 


-127' 


Aptitude V 


86.8 




68- 


111 


92. U 


11.0 


63 


-117 


Aptitude N 


85.9 


16.1 


. 58- 


127. 


90.6 


16. 0 


56 


-136 


Aptitude S 


87.0 


15.1 


58- 


12U 


91.7 


15,9 


61 


-12U 


Aptltude^ 


96.9 


20.1 


,56- 


1U3 


97.U 


17-. 3 


59 


-13U 


Aptitude Q 


107.5 


U.3 


8U- 


1U8 


106.7 


Ik.l 


80 


-151. , 


Aptitude K 


110.8 


20.3 


68- 


159 


99.3 


17.8 


(t9 


-15U 


Aptitude F 


99.8 


18.2 


58- 


1U3 


98.1 


21.9 


37 


-151 


Aptitude M 


105.6 


■ 22.6 


'1*7- 


152 . 


92.2 , 


25.5 


32 


-138 


•'Cr I ter ion 


^ 1*8.7 


11.5 


20- 


70 


51.6 


10. l» 


20 


-70 


Age 


. 26.1. 


10.9 


17- 


5k 


53.1 


13\9 


. 17 


-63 


Education 


10.6 


■ 1.7 

* 


6- 


12' 


I'O.l 


1.7 


7 


-12 
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■^APPENDIX 2 ' , . t 

Hpspltals Contributing Samples fer Vall'datlofi Study 

■Bristol M«no!: I a-1 Hospital, Bristol, Vlrgln'la 
Levfis Gayle HospUal, Roanok«; Virginia. , ^ 

Piedmont Sanatorl»um, Crewt, Virginia ' " . 

Hospitals Contributing Samples for Cross-val Idatloh Study #1 

Presby^ervlari' Hospital/ Phi ladelphia, Pennsylvania 

Salt Lake General Hospital, Sal t Lake CI ty/ Utah 

SAint Mark's Hospital, SSI t Lake City, Utah , ' 

Latter-Day Saints Hospital, Salt Lake City, Utah 

Hospitals Contributing Samples for Cross-validation Stydy #2 

Los Angeles County General,' Los Angeles, Cal Ifornl a , 
Harbor General Hospital, Los Angeles, California 
John Wesley Hospital, Lbs Angeles, 'Cal If orrt la 
Long Beach ^General Hofspltal, Los Angeles, Cal ifornia 
Olive VlewHospltal, Los Angeles, California . 
Rancho Los Amlgos Hospital, Los Angeles, California 



i 



17 



' 1$ - ^ 

' appendix's 

Val tdatlon Sample 



RATING TRAINEES 

ft*- • . 

SUGGESTIONS TO RATERS . " 

We are asking you to rate the job performance of the trainees whom you instructed. These ratings 
will serve as a ^'yardstick" against which we can compare the test scores in this study. The ratings 
must give a4rue picture of eadi trainee or this study will have very little value. You should strive to 
give the' most accurate ratings possible for each trainee* 

These ratings are strictly cop^fidential and Won*t affect your trainees in any way. Neither the ratings 
nor test scores any trainee will be shown to anyone other than personnel of the E$C testing section. 
We BXt interested in only ^testing the tests." Ratings are needeci for only those trainees who are in 
the test study. * . 

>In making ratings, dpn't let general impro6sions or some outstanding trait affect your judgment* Try 
to forget your personal feelings about the trainee. Rate hhrn only on his performance. Here are some 
additional points which might help you: 

f 

1. Please read and study all directions and the rating scale thoroughly before rating a trainee* 

2. For each question compare your trainees with ''trainees in general" for this type of vocational' 
training. We want the ratings to be based on the samestandards in all training courses covering 
the same occupation. ' * * , ' 

3. ' A suggested method is to rate all trainees cn one question at a time. The questions pertain Co the^ 

different abilities of the trainees. A trainee may be good'^in one ability and poor m another; for 
exkmple, a very %low trainee may be very Accurate. So rate all trainees on the first question, then 
rate all trainees on the second qbestion, and so on. 

4. Rate the trainees according to the work they Have done throughout the entire vocational training 
course. Don't rate just on the basis of one "good" day, one "bad*' <^ay of some single incident 
Think in terms of each trainee's usual or typical day by day performance. 

5. Ra|e only on the abilities listed on the rating sheet. Do not let factors such as cooperativeness, 
ability to get along with others, promptness and honesty influence your ratings. Althoughihese 
aspcctsvof a worker are important, they are o^ no value for this study as a "yardstick** against 
whicI^to compare aptitude test scores. - • 



Please fill in4he information requested below* 



RATED BY- 



TITLEL 



LOCATION OF TRAINING- 



(Sum) 



DATEL 
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UNITED STATES EMPLOYMENT SERVICE 
DESCRIPTIVE (RATING SCALE FQR TRAINEES 

"W 

(For Trainees Used in Aptitude Test Development Studies) 
^* • ' Score 



RATING SCALED FOR . 

^ Directions: Please ^eail ^"the suggestions to,raters" on the back (if this form then complete this rating 
scale. In making your ratings, only one box should be checked for each question. 

Name of trainee (print) ^ 

Sex: Male Female 

A. How much aptitude or facility does he have for the vocational training? (Trainee^s adept ness or 
knack for performing the work easily and well.) , , ' , 

□ L Has great diffiCuhy doing^the .work. Not ft all Suited for the training. 

□ 2. U9ually has some diffiauhy doing the work. Not too well suited for the training. 



□ 3, Does the work without loo much difficulty. Fairly well suited for the training. 



□ 4. Usually does the work without difficulty. Well suited for the training. 

□ 5. Does the work with great ease. Exceptionally well suited for the training. ^ 
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• * 

B. HoW much ability does he have for maintaining adequate production' in the vocational activity 
for which he was trained? ^ * , ' 

□ 1. Capable of very low work output. Can perform only at* an unsatisfactory pace. 

□ 2. Capable of low work output. Can perform at a slow pace. 

^ . □ 3.* Capable of fair work output. CanCpi^tforni at an acceptable but not a fast pace. 

□ 4. Capable of high work output. Can perform at a fast pace.' 

□ 5. Capable of very high work output. Can perform at an unusually fast pace^. 



C. HoW^ood was the quality of his woHc^diiring"tKe vocational training? 

□ 1. Performance was inferior and a^^most never met . minimum quality standards. 

□ 2. iPerformancfc was usually acceptable but somewhat inferior in quality. The grade of his 

work could Stand'improveme.it. , * * ^ 

□ 3. Performance wis accept^ible but usually not superior in quality. 

□ 4i Performance was usually superior' in~tquality> 

D 5. Performance was almost always of the highest quality. 



D. Ho^ quickly did he learn the instructional units of the vocational training? 

□ \. Learned the work very slowly. Needed careful and repeated instructions. 

□ 2. Learned'the work somewhat slower tbah'most. 

□ 3, Learned mbst of the work in the usual amount of time. 

□ 4. Learned mosf of the work quickly. • 



□ 5. Learned all of the work very rapidly. Needed only the minimum amount of training or in- 
structions for even the- difficult aspects. 
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\ — : ^ 

E How inuch ability does he have for using the equipment oif the vocational tr^itiing? 

\ ' ■ "V ^\ 

Q 1. H,as very limited ability.»Cannot use the equipment adequately. \ 

□ 2. Ht^t little ability. Can use the equipment to "get by." * ^ * 

□ 3. Hak a moderate amount of ability. Can use the e<|(iipmenl to do faW work. ^ ^ 

□ 4. Has\high ability. Can use the equipment to do good work. ' ^ ' 
ta* 5. Hat Very high ability. Can use the equipment to do^cellent work. ' » ' 

F. ^ How large a waricty of job duties can he perform efficiently? ^ ^ * * 

□ 1. Cannot perform diRerent operations adequately. ^ , 

□ 2. Can perform a limited number of diffefent operations efficiently. 

□ 3. Can perform several different operations with reas^able efficiency. 

□ 4. Can perform many different oprntion^ efficiently. n 
Q 5. Can perform an unusually large vat'kty of different operations efficiently. 

"C, Considering all the factors already rated, and, on/y thesT^ctors, how^cce{^table was his per* 
form an ce during vocational training? 

□ 1. Performance was unsatisfactory. # * 

□ 2. Performance was not completely satisfactory. * * , / ^ ^ 

□ 3. Performance was satisfactory. ^ ^ # ^ ' , 

□ 4. I^erformance was good. i ^ 

* □ 5. Performance was outstanding. • » » *^ 

( ^ . . . 



i 



21 



Cross-val tdat Ion Sample #1 



SP-25 * \ aITERNATION RANKING FORM 

1/67 

L Ranking Instructions 

a. Lilt all workers that you supervise in the Ieft*Kancl column of the opposite page. 

b. Review the list and .eliminate any individual if: 

(1) You are not sufficiently familiar with his pterfornnance to rank him (new employee, etc!). 

(2) His work is so unique that comp«risonJs impossible. 

c. Now add to the list any other present wprkers in thiai same job which you previously super- 
vised that you know well enough to c^valuate. 

d. From the total list, select the person you consider as ranking highest in the ranking factor. 
(Overall periormance/etc.) Write his name^n the top line (labeled "1— Highest") of the Rank 
Order column, and cross his name off the list. 

e. From the remaning names, select the person you feel ranks lowest, and write his name on the 
line at the bottom of the Rank Order polumn where it says ""1 — Lowest**. Now cross his name 

* off the list. » / ^ 

f. Continue by alternately selecting the next highest and the next lowest individuals remaining 
on the worker list until all names have been crossed out. For example, if you have listed eight 
workers, four of them shoUld appear at the top of the Rank Order colutpn and four at the bottom. 
The middle spaces would be left blank. 4 ^ 

The fact that you rate an individual as lowest does not necessarily mc^an that you regard his perform- 
ance as un^^atisfactory. Your rating merely denotes how you ^vould rank him in relation to others in 
the paAicular group. 

2. Rating Instructions * 

a. Think again about the performance of the individual you* have just ranked on the particular 
ranging factor, v . " * v 

b. Now rate each of the ranked workers using the letter ratings below. Place the appropriate 
letter rating after the^'foreman^s name in the Rank Order column under "'Rating**. (Example: 

. Joe Jones S ~ .) ^ ^ 

Lttttr Rating " , . . 

0 (Outstanding: Performance leaves little or nothing to be desired.) 

£ (Excellent. An unusually competent worker. Performance almott always top 

« no|ch.) ' . 

S+ (Good. A valuable workef. Performance usually superior.) 

S (Satisfactory. A fairly proficient worker. Performance generally acceptable.) 

(Not completely satisfactoiy. Of limited value to the organization. Performance 
marginal) 

U (Definitely unsatisfactory. Would be better off without^him. Performance usually 

not acceptable.) 



7 



\ 



Basis for 
Ranking: 



\ 

Alternatiori^ 

RANKING FORXi 

\ > 



Worker Liat 



\ 



R>nk Order 



Rater's 
Signature. 



l-Highesl- 



2 -Next HighW- 
^3 -Next Highest^ 

4- Next Highei^l— 

5 - Next Higheil-- 

6 - Next Highes\- 

7- Ne)a Highcsi\p 

8- Next Highesi-V 

9 - Next Highest^ 

10 - Next Highest- 

11- Next Highesl- 

12- Next Highest- 

13- Next Highesl- 



14- Next Highest— 
1^— Next Highest — 

16- Next Highest— 

17- Next Highest— 

18- Next Highest— 

18— Next Lowest 

17 — Next Lowest 

16 — Next Lowest 

15 — Next Lowest 

14— Next "Lowest 

13— Next Lowest — 
12— Next Lowesi^^ 

11 — Next Lowest 

10— Next Lowest 

9— Next Lowest-^ 

8— Next Lowest 

7— Next Lowest 

6— Next Lowest 



5— Next Lowest- 
4— Next Loweft- 
3— Next Lowest- 
2— Next Lowest- 
1 — Lowest 



Position^ 



PUat 



DepattiMiit 



Date- 
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Cross-vaMdatron Sample *2 * r; 

SUGGESTIONS TO- RATERS 

We are asking you to rate the job performan^ce of the people who work for you* 
These ratings will ser^ as a *^ardstick*^ against which we cai^ compare the * 
test icores in this study* The ratings must give a true picture of each 
woricer or this study will have very little valpe* You should try to give the * ^ 
most accurate ratings possible for each worker. 

These ratings are strictly confidential and won't affect your Workers in any' r 
Way* Neither the ratings i^fir test ?^cores of any worker will be shown to^ ^ 
anybody in your company* W* are interested <ply in "testing the tests*" 
Ratings are needed only for tho^e worker^ .who are in the test study* 

• • *■ 

Wox^cers who ha<ye not completed their training period,* or ^ho •have not been on 
the job or under your supervision long enough for you to know how well^ they . * 
can perform this work should not be rated* Please inform the test technician 
about this if you are asked to rate any such, workers* , *. 

In making ratings, don't let general impressions or 9ome'l9utstanding trait affect 
your judgment* Try to forget your personal feelings about the worker. Rate 
him only on the way he does his work^'^ Here are* some more points which might ^elp 
you: s . ♦ ^ . * , 

1 

\ 1* Please i^ead^all directions and the rating scale thoroughly before rating ' 

^ 2* For each ^question compare your workers with **workers<:in-g^neral" in this 
job« That is, compare your workers with other workers on this job that 
you have known. This is very important in small plants Where ^here are 
only a few woricers* We want the ratings to be based on the sproe standard 
in all the plants* * • ' ' I * . . ■ 

3* A suggested method is to rate alJi workers on one^ question at m tiltie* The 
questions ask about different abilities of the workers* A worker may be i 
good in one ability and poor in another; for example, a very slow worker 
may be very accurate. So rate all workers on the first question, than 
rate all workers on 'the second question, , and so on, 

4* Practice"^ anti enperience iisually improve a worker's skill* Howevery one 
wo}[*er 'with six months' experience may be a faster worker than another with 
six years' experience. Don't rate one worker as poorer than another 
mirely .because he has not bben on the job ^as long, 

5* Rate th% workers according to the woric they have done over a period of t * 
several weeks or months. Don't rate just on the basis of one "good't ^ 
day, one "bad" day or some single incident. Think in term^ of each 
worker's usual or typical performance, \ 

Rate only on the abilities listed on the rating sheet* Do nc^t let factors 
.such as cooperativeness, ability 'to' get along with others, promptness ^and 
honesty influence your ratings*^ Although thea a;^cts of a worker are ^ 
important, they are of no value^for this study as a "yardstick" against 
which to compare aptitude test scores* ^ 

Please write your nam, title, company, city, state, and date of rating on a 
separate sheet of paper for each^ set . of t^atings you make* 

2i 
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« DESCRIPTIVE liATING SCALE 

' ' ' {fQT GAIB Followup &tudx on Occupational Perfbimnc*) 

. . , , ' ^ 

RATING SCALE PORV \ SOORE_ 

> »T TITLE AND ODK (3rd Hitlon^ • • 



I 



DIRECTIONS: ' Pleas« nad th« shei^t ^'Suggtstions to'IUt«i^'' «d 
*; « thm fill in th«'it«iis lisUd b«low. In making your ratinga.' 
\ * only ona box should bo^djecked fdr aach quastion. 



NAI€ OF WORKER (print) ' 

- . ' nSstT (PiMtJ- 

^ . . . 

SOCIAL SECURITf NUMBER \ SEX: Mala ^Feiiale_ 

OOWANY * . . -LOCATION 



WORKER'S EWERIENCE ON PRESENT JO B . J OB TRAINING PERIOD .__ 

^ (Months) ' (M>nth») 



' RATED »Y T ITLE 1 



I- 



How oftan do you sea this worker in a work situation? 
rn Safa hlB at work all tha'time? . / , ^ 

£37 Saa hlM at work seyaral times a day.. ' / ' 

Saa Kin at work several times a weak. * - . , ' , 

^^^^ ^ « 

/~7 Seldom see him in voxl^ situation. * ' *\ ' 

/ » * 

« • •# • 

How long have you worked with him? « * • T • 

/ 7 * Under ona,;mDnth« • ^ . ^ * 

One to two n>nths« • . . 

Ihree to five months* * ' . 

/ / Six months or mora. < * » ^ 

♦ ES Analyst should assign DOT Title and Code 0n the basis of a discussion of the 
job duties with the Supervisor ^ ' o 
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A. How Buch work .can ht get doiit? (Workeir^s ability to mikt tfflcient U83*of 
his tlrot and to woric at high apetdt) ^ V « 

• ♦ • ♦ 

/ 7 1. Capablt of very low work, output, , Can perfoxa only at an unsatia* 
J factoxy pact* v- , • 

/ jf 2. Capablt of low wofk output. Can perfon at a slow pact. 

/~7 3. Capablt of fair wortc output. Can ptrt?6na at an acctptablt but not * 
a faat pact, 

/ y Capable of high woxk output « Cap ptrfoz^ ajt a fast pact* 

5. Capable of very high work outputs Can perform at a^ unuaually fiat 
pact. i 

♦ • 

B. How good is thtf quality of hit work? (Worktr^a ability to. do bigb-gradt work 
which Bttta quality eiandarda.) 

/ / 1. Ferfoxnance la inftrior and alaoet never meets ^niniaiua* quality • 
standards. ' ^ ^ » 

2. gr4de of his work cpuld stand improvenent. Performance is usually 
\ Acceptable* but somewhat inferior in quality^j^ 

/ y 3. Perfoxmancte is acceptable but usui^Cly not superior in qualilgr. 

/ y^4> . Performance is usually superior in quality. 

* 5\ -Perfoimance^ia. almost always of the highest quality. ^ 

C. How accurate is he in his work? (Worker's ability to aroid making mistake^*) 

1. Makes' Tsx/ many miatakes. Work needs coiatant checking. 
/ 7 2. Makes ^quent mistakes. Work nsede score checking than is desirable* 

7^ 3. Makes mistakea occasionally « Voxk needs only noxmal ohedding. 

- / ' 

^. Makei few mistakes.- Work seldom needs cheddng. 

y 7 5.- B*r«l7 mak«a a alstakt. ''Wozk aljiost nertr netda ch0dd.nf;* 



ERIC 
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How much dots tm know^bout his job? (Woilcfjft^i!; ujiderslkri^^ of ths prlnelplss, 
oquipaent. Mteil«ls-^ananB5t5io* to do dlrsotly or indlrt oily with 

his work^) • ♦ • " 

/ y !• Km rery limited knowledge. Boos not know enough to do his job 
adequately* 

2. Has little knowledge^ Knows enough to •♦^et by." 

3* Has moderate amount of knowledge. Knows enough to do fair work, 

/ '/ A J Has broad knowledge. Knows enough to do good work* . 

5. Has cooplste knowledge. Knows his job thoroughly. 

How much aptitude or facility does be hare for this kind of. work? (forker*s 
adeptneas or knack" for perfoxming his job easily and well*) 

/. / 1. Has great difficulty doing his job. Hot at all suited to this kind 
of wozk. 

2, Usually has soae diffictdty doing his job. Hot too well suitsd'to 
this kind of work. 

____ # * 

3. Ibes his -job without too much difficulty." Pairly well suited to this 

kind of work. 

m 4. Usually does his job without difficulty. Well suited to this kind 
of work. ' * 

/~7 5. loes his job with great ease. Kxcoptionally well sui^ted }Mr this 
, kind of wozk* - - \^ ' 

How large a yariety of ^ job duties can he perfoza efflcientlyt (Worker's ^ 
ability to handle ssVeral different operations in his work.) 

f / l4 .Cannot perfora different operations adequately* 

^ ] 2. Can perform a limited number of different operations effiolenilX# 

% Can perform sersral diffsrent opsrations with reasonable sffldencx* 

/ / 4« Can perform oapy different operations efficiently* 

J / % Can psrfoxn an unusually large rariety of diffsrent operations 
efficiently. 



0« How rtMure^ful Is h% when Miethixtg different eoiee up or sosething oat of 
tbm ordinary occurs? (Worker's ability to apply what he already knows to a 
, new situation. ) 

/ / 1* Also St neyer Is able to figure out what to do. Veeds help on eren 
oinor problems. ^ ' ' 

/~7 2. Often has difficulty handling \new situations. Needs help on ail but 
. . ^ siaple problems. 

/ 7 3* Sometiities knows what to do,/ 8om)i^ times doesnH* Can deal with problems 
that are not too complex* ^ 

/ / 4* Usually able to handle new ei^tions. Heeds help on' only complex 
problems. . 

/ y 5. Practically e^iways figures out what to do himself • Barely needs 

helpy even on complex problems. * * , 

How maiqr practical suggestions does he make for doing things in better ways? p 
* (Worker > ability to improTe work methods.) f 

* 1. Sticks strictly with the routine. Contributes nothing In the way 
of practical suggestions. 

/ / 2. Slow to see new ways to improre methods. Contributes few practical ^ 
suggestions. ^ 

/ / 3. Neither quick nor slow to see new ways to improTe methods. Contributes 
some practical suggestions. ^ 

/ / 4« Quick to see new ways to improTe methods. Contributes more than his 
share of practical suggestions. 

/ / 5, Ebctremely alert to see new ways to improve methods. Contributes an 
unusually large number of practical suggestions. 

Considering all the factors already rated/ and only these factors, how acceptable 
is his work? (Worker's "all-around" ability to do his job.) 

/~7 1. Would be better off withou|, hin^. Performance usually not acceptable'. 

/ / 2.' Of limited value to the orgemixation. Performance somewhat inferior^ 

/~7 3. A fairly proficient worker. Performance generally acceptable. 

jTy 4. A vaiuable woricer. Performance usually superior. < 

/ / 5, An unusually competent worker. Perfoimance almost always, top notch. 
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Nurse Aid (medical s,er. ) J5'5.878 
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JOB DUTIES 

Provides routine personal and nurslfig care to patients^ 
rf^alntalns and Issues supplies and equipment and performs related 
clerical duties underthe direction of nursing and medical staff 

♦Provides personal ca^re to patients fpl lowing daN assl gnment ^ 
sheet and verbal Instructions: Escorts or transports patients 
to room or treatment unit using wheelchairs and stretchers. 
Lists' and stores pa.tlents' clothes'and possessions. Bathes^ 
dresses and undresses patl ents. Serves and collects food trays\ 
and fepds patients requiring help. Provides drinking water and^ 
nourishment between meals. Dusts and cleans patient's, room. 
Changes bed linen. Assists patients with bedpans and urinals. 
Cleans bedpans' and urInaVs. Removes tr'as^ such as sputum oups 
and refuse bags from bed^slde. , • * 

< 

♦Provides nursing care to patients following dally assignment 
sheet and verbal Instructions': Answers .si gna*l lights and bells 
and determines patl.ent's needs. Observes ^>a^t lent and reports 
unfavorable conditions to nursing or medical s^aff. Takes and 
records patient's blood pressure^ temperature/ pulsc^ and 
respiration. Collects specimens'. Aamlnlsters dbuches and 
enemas. ^Records solid and Itquld Intake and ^output. JVpplies 
hot or^cold treatmenr. Assists In ex^ml n'^Tt Ion and * treatment- by 
performing such duties as drap! ng^ P^tlentS/ holding Instruments 
and adjusting lights. , 

* * 

♦Maintains land Issues supplies and equipment: Cleans^ 
<;steri 1 izes^ and stores equipment and supplies. Prepares and 
Issues treatment trays and dressl ng^ packs. Maintains or assists 
In maintaining perpetual Inventory of supplies. * 

Performs rejated cledcal duties: Schedules and records 
appoljitments. Retrieves and files rjecord folders. Answers 
• telephone and deTlvers messages. 



♦These duties were designated as critical because they must be 
'performed competently If the job Is to be done In a satisfactory 
manner. Nurse Aids spend about 75!^ of their working hours per^ 
forming these job duties. 
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